Skip to content

Add tags to invokers. #5289

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Jul 22, 2022
Merged

Add tags to invokers. #5289

merged 4 commits into from
Jul 22, 2022

Conversation

style95
Copy link
Member

@style95 style95 commented Jul 21, 2022 •
edited
Loading

Description

This is to enable tag-based scheduling.
We can add tags to invokers like below.

[invokers]
invoker0            ansible_host=172.17.0.1 ansible_connection=local tags="['v2', 'gpu']"

There are already some codes for tag-based scheduling.
https://github.com/apache/openwhisk/blob/master/tests/src/test/scala/org/apache/openwhisk/core/scheduler/container/test/ContainerManagerTests.scala#L615

This is to add missing parts.

Related issue and scope

  • I opened an issue to propose and discuss this change (#????)

My changes affect the following components

  • API
  • Controller
  • Message Bus (e.g., Kafka)
  • Loadbalancer
  • Scheduler
  • Invoker
  • Intrinsic actions (e.g., sequences, conductors)
  • Data stores (e.g., CouchDB)
  • Tests
  • Deployment
  • CLI
  • General tooling
  • Documentation

Types of changes

  • Bug fix (generally a non-breaking change which closes an issue).
  • Enhancement or new feature (adds new functionality).
  • Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

  • I signed an Apache CLA.
  • I reviewed the style guides and followed the recommendations (Travis CI will check :).
  • I added tests to cover my changes.
  • My changes require further changes to the documentation.
  • I updated the documentation where necessary.

@ningyougang
Copy link
Contributor

ningyougang commented Jul 21, 2022 •
edited
Loading

  • Can add some example of how to user it? e.g. If my action wants to use special invokers(e.g. include TAG: cpu in ansible), my action should meet what condition?
  • Can use in kubernetes?

@ningyougang
Copy link
Contributor

LGTM with some small questions.

@codecov-commenter
Copy link

codecov-commenter commented Jul 21, 2022 •
edited
Loading

Codecov Report

Merging #5289 (3f6efcb) into master (a03507c) will decrease coverage by 4.67%.
The diff coverage is 91.66%.

@@            Coverage Diff             @@
##           master    #5289      +/-   ##
==========================================
- Coverage   79.90%   75.23%   -4.68%     
==========================================
  Files         238      238              
  Lines       14161    14172      +11     
  Branches      603      624      +21     
==========================================
- Hits        11316    10662     -654     
- Misses       2845     3510     +665
Impacted Files Coverage Δ
...la/org/apache/openwhisk/core/invoker/Invoker.scala 73.33% <90.90%> (+2.56%) ⬆️
.../scala/org/apache/openwhisk/core/WhiskConfig.scala 95.97% <100.00%> (+0.02%) ⬆️
...core/database/cosmosdb/RxObservableImplicits.scala 0.00% <0.00%> (-100.00%) ⬇️
...ore/database/cosmosdb/cache/CacheInvalidator.scala 0.00% <0.00%> (-100.00%) ⬇️
...e/database/cosmosdb/cache/ChangeFeedConsumer.scala 0.00% <0.00%> (-100.00%) ⬇️
...core/database/cosmosdb/CosmosDBArtifactStore.scala 0.00% <0.00%> (-95.85%) ⬇️
...sk/core/database/cosmosdb/CosmosDBViewMapper.scala 0.00% <0.00%> (-93.90%) ⬇️
...tabase/cosmosdb/cache/CacheInvalidatorConfig.scala 0.00% <0.00%> (-92.31%) ⬇️
...enwhisk/connector/kafka/KamonMetricsReporter.scala 0.00% <0.00%> (-83.34%) ⬇️
...e/database/cosmosdb/cache/KafkaEventProducer.scala 0.00% <0.00%> (-78.58%) ⬇️
... and 16 more

@style95
Copy link
Member Author

style95 commented Jul 21, 2022 •
edited
Loading

I confirmed it's working as expected.
I wl add documentation about this as well.

OW operators can configure tags for invokers like this.

invoker0 ansible_host=${INVOKER0} tags="['v1', 'gpu']"
invoker1 ansible_host=${INVOKER1} tags="['v1', 'cpu']"
invoker2 ansible_host=${INVOKER2} tags="['v2', 'gpu']"
invoker3 ansible_host=${INVOKER3} tags="['v2', 'cpu']"
invoker4 ansible_host=${INVOKER4} tags="['v1', 'mem']"
invoker5 ansible_host=${INVOKER5} tags="['v2', 'mem']"
invoker6 ansible_host=${INVOKER6} tags="['v2']"
invoker7 ansible_host=${INVOKER7} tags="['v2']"
invoker8 ansible_host=${INVOKER8}
invoker9 ansible_host=${INVOKER9}

Users can add the following annotations to their actions.

wsk action update params tests/dat/actions/params.js -i -a invoker-resources '["v2", "gpu"]'

Activation for this action will be delivered to invoker2.

The annotations and the corresponding target invokers are as follows.

  • ["v1", "gpu"] -> invoker0
  • ["v2", "gpu"] -> invoker2
  • ["v1", "cpu"] -> invoker1
  • ["v2"] -> One of invoker2, invoker3, invoker5, invoker6, and invoker7
  • ["v1"] -> One of invoker0, invoker1, invoker4
  • No annotation -> One of invoker8 and invoker9 is chosen first. if they have no resource, choose one of the invokers with tags.

style95
style95 commented Jul 21, 2022
View reviewed changes
core/invoker/src/main/scala/org/apache/openwhisk/core/invoker/Invoker.scala

logger.info(this, s"invoker tags: (${tags.mkString(", ")})")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this.

[2022-07-21T11:00:02.599Z] [INFO] [#tid_sid_unknown] [Invoker] invoker tags: (v1, cpu)

@style95
Copy link
Member Author

style95 commented Jul 22, 2022

It is required for the subsequent change.
Let me merge this for now.
Please feel free to leave any comments. I would apply them in the subsequent PR if anyone has.

@style95 style95 merged commit d55d8fe into apache:master Jul 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ningyougang ningyougang ningyougang approved these changes

@jiangpengcheng jiangpengcheng Awaiting requested review from jiangpengcheng

@bdoyle0182 bdoyle0182 Awaiting requested review from bdoyle0182

@KeonHee KeonHee Awaiting requested review from KeonHee

@upgle upgle Awaiting requested review from upgle

Assignees
No one assigned
Labels
scheduler
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@style95 @ningyougang @codecov-commenter